347 research outputs found
A Neural Attention Model for Categorizing Patient Safety Events
Medical errors are leading causes of death in the US and as such, prevention
of these errors is paramount to promoting health care. Patient Safety Event
reports are narratives describing potential adverse events to the patients and
are important in identifying and preventing medical errors. We present a neural
network architecture for identifying the type of safety events which is the
first step in understanding these narratives. Our proposed model is based on a
soft neural attention model to improve the effectiveness of encoding long
sequences. Empirical results on two large-scale real-world datasets of patient
safety reports demonstrate the effectiveness of our method with significant
improvements over existing methods.Comment: ECIR 201
Label-Dependencies Aware Recurrent Neural Networks
In the last few years, Recurrent Neural Networks (RNNs) have proved effective
on several NLP tasks. Despite such great success, their ability to model
\emph{sequence labeling} is still limited. This lead research toward solutions
where RNNs are combined with models which already proved effective in this
domain, such as CRFs. In this work we propose a solution far simpler but very
effective: an evolution of the simple Jordan RNN, where labels are re-injected
as input into the network, and converted into embeddings, in the same way as
words. We compare this RNN variant to all the other RNN models, Elman and
Jordan RNN, LSTM and GRU, on two well-known tasks of Spoken Language
Understanding (SLU). Thanks to label embeddings and their combination at the
hidden layer, the proposed variant, which uses more parameters than Elman and
Jordan RNNs, but far fewer than LSTM and GRU, is more effective than other
RNNs, but also outperforms sophisticated CRF models.Comment: 22 pages, 3 figures. Accepted at CICling 2017 conference. Best
Verifiability, Reproducibility, and Working Description awar
Using Regular Languages to Explore the Representational Capacity of Recurrent Neural Architectures
The presence of Long Distance Dependencies (LDDs) in sequential data poses
significant challenges for computational models. Various recurrent neural
architectures have been designed to mitigate this issue. In order to test these
state-of-the-art architectures, there is growing need for rich benchmarking
datasets. However, one of the drawbacks of existing datasets is the lack of
experimental control with regards to the presence and/or degree of LDDs. This
lack of control limits the analysis of model performance in relation to the
specific challenge posed by LDDs. One way to address this is to use synthetic
data having the properties of subregular languages. The degree of LDDs within
the generated data can be controlled through the k parameter, length of the
generated strings, and by choosing appropriate forbidden strings. In this
paper, we explore the capacity of different RNN extensions to model LDDs, by
evaluating these models on a sequence of SPk synthesized datasets, where each
subsequent dataset exhibits a longer degree of LDD. Even though SPk are simple
languages, the presence of LDDs does have significant impact on the performance
of recurrent neural architectures, thus making them prime candidate in
benchmarking tasks.Comment: International Conference of Artificial Neural Networks (ICANN) 201
Discrete and fuzzy dynamical genetic programming in the XCSF learning classifier system
A number of representation schemes have been presented for use within
learning classifier systems, ranging from binary encodings to neural networks.
This paper presents results from an investigation into using discrete and fuzzy
dynamical system representations within the XCSF learning classifier system. In
particular, asynchronous random Boolean networks are used to represent the
traditional condition-action production system rules in the discrete case and
asynchronous fuzzy logic networks in the continuous-valued case. It is shown
possible to use self-adaptive, open-ended evolution to design an ensemble of
such dynamical systems within XCSF to solve a number of well-known test
problems
Recommended from our members
Words, rules, and mechanisms of language acquisition
We review recent artificial language learning studies, especially those following Endress and Bonatti (2007), suggesting that humans can deploy a variety of learning mechanisms to acquire artificial languages. Several experiments provide evidence for multiple learning mechanisms that can be deployed in fluent speech: one mechanism encodes the positions of syllables within words and can be used to extract generalization, while the other registers co-occurrence statistics of syllables and can be used to break a continuum into its components. We review dissociations between these mechanisms and their potential role in language acquisition. We then turn to recent criticisms of the multiple mechanisms hypothesis and show that they are inconsistent with the available data. Our results suggest that artificial and natural language learning is best understood by dissecting the underlying specialized learning abilities, and that these data provide a rare opportunity to link important language phenomena to basic psychological mechanisms
Neurodevelopmental disorders
Recent technological advances allow us to measure how the infant brain functions in ways that were not possible just a decade ago. Although methodological advances are exciting, we must also consider how theories guide research: what we look for and how we explain what we find. Indeed, the ways in which research findings are interpreted affects the design of policies, educational practices, and interventions. Thus, the theoretical approaches adopted by scientists have a real impact on the lives of children with neurodevelopmental disorders (NDDs) and their families, as well as on the wider community. Here, we introduce and compare two theoretical approaches that are used to understand NDDs: the neuropsychological account and neuroconstructivism. We show how the former, adult account, is inadequate for explaining NDDs and illustrate this using the examples of Williams syndrome and specific language impairment. Neuroconstructivism, by contrast, focuses on the developing organism and is helping to change the way in which NDDs are investigated. Whereas neuropsychological static approaches assume that one or more ‘modules’ (e.g., visuospatial ability in Williams syndrome) are impaired while the rest of the system is spared (e.g., language in Williams syndrome), neuroconstructivism proposes that basic‐level deficits have subtle cascading effects on numerous domains over development. Neuroconstructivism leads researchers to embrace complexity by establishing large research consortia to integrate findings at multiple levels (e.g., genetic, neural, cognitive, environmental) across developmental time
A framework for Distributional Formal Semantics
Formal semantics and distributional semantics offer complementary strengths in capturing the meaning of natural language. As such, a considerable amount of research has sought to unify them, either by augmenting formal semantic systems with a distributional component, or by defining a formal system on top of distributed representations. Arriving at such a unified framework has, however, proven extremely challenging. One reason for this is that formal and distributional semantics operate on a fundamentally different `representational currency': formal semantics defines meaning in terms of models of the world, whereas distributional semantics defines meaning in terms of linguistic co-occurrence. Here, we pursue an alternative approach by deriving a vector space model that defines meaning in a distributed manner relative to formal models of the world. We will show that the resulting Distributional Formal Semantics offers probabilistic distributed representations that are also inherently compositional, and that naturally capture quantification and entailment. We moreover show that, when used as part of a neural network model, these representations allow for capturing incremental meaning construction and probabilistic inferencing. This framework thus lays the groundwork for an integrated distributional and formal approach to meaning
Look and Feel What and How Recurrent Self-Organizing Maps Learn
International audienceThis paper introduces representations and measurements for revealing the inner self-organization that occurs in a 1D recurrent self-organizing map. Experiments show the incredible richness and robustness of an extremely simple architecture when it extracts hidden states of the HMM that feeds it with ambiguous and noisy inputs
Ant-based Neural Topology Search (ANTS) for Optimizing Recurrent Networks
Hand-crafting effective and efficient structures for recurrent neural networks (RNNs) is a difficult, expensive, and time-consuming process. To address this challenge, we propose a novel neuro-evolution algorithm based on ant colony optimization (ACO), called Ant-based Neural Topology Search (ANTS), for directly optimizing RNN topologies. The procedure selects from multiple modern recurrent cell types such as ∆-RNN, GRU, LSTM, MGU and UGRNN cells, as well as recurrent connections which may span multiple layers and/or steps of time. In order to introduce an inductive bias that encourages the formation of sparser synaptic connectivity patterns, we investigate several variations of the core algorithm. We do so primarily by formulating different functions that drive the underlying pheromone simulation process (which mimic L1 and L2 regularization in standard machine learning) as well as by introducing ant agents with specialized roles (inspired by how real ant colonies operate), i.e., explorer ants that construct the initial feed forward structure and social ants which select nodes from the feed forward connections to subsequently craft recurrent memory structures. We also incorporate communal intelligence, where best weights are shared by the ant colony for weight initialization, reducing the number of backpropagation epochs required to locally train candidate RNNs, speeding up the neuro-evolution process. Our results demonstrate that the sparser RNNs evolved by ANTS significantly outperform traditional one and two layer architectures consisting of modern memory cells, as well as the well-known NEAT algorithm. Furthermore, we improve upon prior state-of-the-art results on the time series dataset utilized in our experiments
Hybrid of swarm intelligent algorithms in medical applications
In this paper, we designed a hybrid of swarm intelligence algorithms to diagnose hepatitis, breast tissue, and dermatology conditions in patients with such
infection. The effectiveness of hybrid swarm intelligent algorithms was studied since
no single algorithm is effective in solving all types of problems. In this study, feed forward and Elman recurrent neural network (ERN) with swarm intelligent algorithms
is used for the classification of the mentioned diseases. The capabilities of six (6) global optimization learning algorithms were studied and their performances in training as well as testing were compared. These algorithms include: hybrid of
Cuckoo Search algorithm and Levenberg-Marquardt (LM) (CSLM), Cuckoo Search algorithm (CS) and backpropagation (BP) (CSBP), CS and ERN (CSERN), Artificial Bee Colony (ABC) and LM (ABCLM), ABC and BP (ABCBP), Genetic Algorithm
(GA) and BP (GANN). Simulation comparative results indicated that the classification accuracy and run time of the CSLM outperform the CSERN, GANN, ABCBP,
ABCLM, and CSBP in the breast tissue dataset. On the other hand, the CSERN performs better than the CSLM, GANN, ABCBP, ABCLM, and CSBP in both th
- …